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Teamworking  to  Develop  Technology  Strategy 

Martin  J.  Read  &  Anthony  E.  Gear 
Balliol  College,  Oxford 

".  ,  .  gather  all  the  food  of  those 
good  years  that  come  .  .  .  and  that 
food  shall  be  store  to  the  land 
against  the  seven  years  of  famine 

."  Joseph,   Genesis  Ch.   41, 
verses  34-36. 

1    Introduction 

The  strategy  of  an  enterprise  is  embodied  in  its  strategic 
decisions,  which  are  basic  to  it  because  "they  deal  with  the 
direction  of  the  enterprise,  its  future  size  and  pattern  of  outputs 
and  markets,  .  .  .  they  determine  the  kind  of  company  which  has  to 
be  managed  and  the  kinds  of  talents  which  are  called  for."  (Minkes 
[1]). 

Technology  Strategy  is  concerned  with  decisions  to  develop, 
exploit,  or  maintain  the  totality  of  organizational  know-how  in  a 
defined  set  of  technologies.  Relevant  activities  in  this 
connection  are  identifying  and  evaluating  technologies  in  terms  of 
their  potential  importance  to  overall  strategy.  The  paper  is 
concerned  with  assessing  the  feasibility  and  significance  of 
various  tabled  technology  development  options  in  a  team  setting. 
The  paper  describes  a  group  decision  support  system  (GDSS)  known  as 
TEAMWORKER,  designed  to  facilitate  forward  planning  by  a  team,  and 
demonstrates  its  use  by  means  of  a  case  application  involving 
technological  forecasting  for  the  time-frame  2010  to  2030. 


The  literature  on  technology  assessment  and  forecasting  is  large. 
For  reviews  of  r.anagenent  techniques  in  these  fields  see,  for 
example,  Jantsch  [7],  Blake  [2],  Souder  [13]  and  Ford  [4].  Some 
techniques  seek  to  make  use  of  the  collective  intellect  of  an 
assembled  group  of  "experts"  rather  than  that  of  a  single  "expert", 
by  means  of  juxtaposing  disciplines  and  experiences  in  order  to 
promote  debate,  creativity  and  intuition.  The  philosophy  of  this 
paper  is  based  on  the  use  of  collective  expertise  to  assess  future 
technological  possibilities  in  a  feedback  oriented  approach,  Gear 
[6],  Computer  based  technology  is  utilized  to  promote  group 
communication,  and  is  carefully  designed  to  minimize  deleterious 
effects  of  conformity,  domination,  reticence,  indiscipline, 
deviation,  reconditeness  or  vested  interest. 

The  TEAMWORKER  GDSS  comprises  hardware  and  software  designed  for 
the  particular  group  context  and  process,  and  is  described  in  a 
variety  of  applications  by  Gear  15,6]  and  Read  ]10,  11].  In  the 
case  described,  the  software  (sometimes  known  as  "Groupware")  is 
aimed  at  improving  communication  for  technology  assessment. 
Mockler  [8],  and  Finlay  [3],  have  recently  reviewed  various  forms 
of  groupware  and  GDSS  for  aiding  strategic  management  processes. 
TEAMWORKER  software  is  aimed  at  facilitating  group  dialogue 
following  feedback  of  judgements  from  group  to  individual  and  from 
individual  to  individual.  This  aspect  of  communication  is  also 
discussed  in  [17]. 


The  software  may  be  designed  around  alternative  frameworks  of 
decision  analysis  [6].  For  example,  [9]  adopted  a  pairwise 
comparative  framework,  based  on  a  two-level  hierarchy,  in  order  to 
rank  a  set  of  alternative  technologies  to  meet  a  defined  need.  A 
critical  survey  of  recent  developments  in  applied  multiple  criteria 
decision  making  and  their  current  status  has  been  undertaken  [12]. 
This  work  provides  a  useful  input  to  the  design  of  groupware,  given 
that  it  is  the  group  communication  process  which  is  of  central 
concern  rather  than  mathematical  routines  alone. 

In  the  case  described  in  this  paper,  a  simple  scoring  scale  was 
made  the  basis  of  individual  judgements  throughout  the  meeting. 
This  meant  that  rapid  interactions  were  easy,  with  the  ability  to 
repeat  stages,  formulate  additional  items,  and  respond  to  new 
requests  "on  the  fly".  (see  also  [14]  in  the  connection).  For  the 
given  purpose,  technology  assessment  on  long  time-frames, 
refinement  of  the  technique  is  not  the  main  issue.  As  Blake  [2] 
has  pointed  out:  "Over-refinement  of  the  system  makes  it  possible 
to  obscure  issues  with  masses  of  detail,  in  turn  causing  error  in 
forecasting".  Complex  approaches,  while  of  considerable  interest, 
may  be  totally  unsuitable  for  certain  business  uses. 

The  underlying  management  technique  behind  the  on-line  system, 
TEAMWORKER,  in  application  to  technology  assessment,  is  GROUP 
DELPHI.  Some  of  the  advantages  of  GROUP  DELPHI  over  and  above 
standard  DELPHI  are  discussed  by  [15]. 


A  further  important  aspect  to  take  into  account  in  the  design  phase 
of  preparing  process  support  software  is  the  cultural  setting  and 
history  of  the  situation,  together  with  the  skills  and  backgrounds 
of  the  participants.  For  example,  [16]  has  described  the  "Emerging 
Technology  Roadmap"  framework  of  thinking,  which  is  updated  by  a 
small  committee  of  individuals  within  Motorola. 

Each  of  the  above  issues  needs  to  be  born  in  mind  in  order  to 
design  a  framework  for  the  group  process  which  can  form  the  basis 
of  group  support  software.  In  Section  3,  a  case  study  is  described 
in  order  to  illustrate  design  considerations.  The  group  decision 
support  system  (GDSS)  used  in  this  application  is  described  in 
Section  2,  and  some  conclusions  are  identified  in  Section  4. 


2.    TEAMWORKER  SYSTEM 

The  system  comprises  a  set  of  hand-held  devices,  one  for  each  group 
member.  Digital  signals  from  each  unit  are  transmitted  to  a  remote 
receiver,  and  fed  into  a  micro-processor.  The  signals  are  analyzed 
in  accordance  with  the  software  which  is  being  used.  Processed 
information  is  displayed  on  a  large  screen  visible  to  the  group. 
The  screen  is  used  to: 

request  information  from  each  group  member  during  the 
meeting. 


display  processed  feedback  information  at  a 
number  of  stages. 

provide  expert  advice  and  menu  options  during 
the  meeting. 

The  arrangement  of  the  system  of  feedback  is  shown  in  Figure  1. 

Certain  judgmental  processes  are  especially  useful,  depending  on 
the  issues  involved.   Some  important  ones  are  itemized  below: 

(i)    Voting 

Uses  include  selecting  an  item  or  option  from  a  set, 
and  multiple  choice  questions. 

(ii)   Scoring 

Uses  include  evaluating  each  of  a  set  of  items  or 
options  on  a  pre-defined  scale,  scoring  as  a  means  of 
producing  lists  in  rank  order,  scoring  as  a  means  of 
quickly  filtering  long  lists  to  short-lists  for  closer 
attention. 


(iii)  Comparing 

Uses  include  weighting  the  relative  importance  of  a  set 
of  criteria  by  means  of  pairwise  comparison,  evaluating 
subjective  factors  and  parameters,  assessing  subjective 
probabilities  and  risks. 

( iv)   Direct  Assessment 

Uses  include  the  estimation  of  parameters, 
probabilities,  risks  and  other  factors  by  direct  and 
subjectively  based  input  values. 

The  system  design  facilitates  communication  at  a  meeting  in  support 
of  group  work  on  four  related  activities: 

(i)    pooling   of   information,   expertise,   opinions   and 
beliefs. 

(ii)   comparison    and    debate    of    areas    of 
agreement /disagreement. 

(iii)  identification  of  key  elements  of  the  decision  task  for 
further  group  attention. 


(iv)   revision   and   redefinition  of   the  task,   options, 
criteria,  etc.  as  the  group  progresses. 

3.  Case  Study;  Technology  Assessment 

Background:  This  case  study  concerns  the  Research  and  Development 
department  of  a  large  multinational  oil  company.  It  is  known  that 
the  future  market  for  petroleum  products  is  uncertain,  owing  to  the 
rising  costs  of  production  and  discovery,  and  the  increasing 
environmental  concerns  associated  with  the  use  of  petroleum 
products. 

These  and  other  issues  led  the  R&D  department  to  consider  the 
science  and  technologies  it  should  be  involved  with  by  the  year 
2010.  This  is  part  of  a  continuing  process  led  by  a  "Core  Team" 
involving  the  study  of  two  key  aspects: 

(a)  "Business  Drivers",  i.e.  Trends  culminating  in 
opportunities  or  threats  to  the  organization 

(b)  "Business  Options",  i.e.  A  set  of  projected 
capabilities  which  would  enable  implementation 
of  a  specific  technology  for  a  defined 
business  purpose  at  some  future  time  period. 


The  organization  decided  that  part  of  this  continuing  process 
should  be  a  three-day  workshop  involving  senior  executives  from  the 
R&D  function  and  from  a  cross-section  of  other  business  functions. 
A  total  of  42  senior  executives  took  part  in  the  workshop,  30  from 
the  R&D  function  and  12  from  other  business  areas. 

The  purpose  of  the  workshop  was  to  exchange  views  and  pool 
judgements  relating  to  the  business  options  being  considered  by  the 
organization,  and  to  provide  guidance  to  the  Core  Team  on  the  high 
grade  options,  the  options  to  drop,  and  the  options  requiring 
further  work.  It  was  stressed  that  this  workshop  was  not  so  much 
a  decision-making  forum,  bur  rather  an  opportunity  to  exchange 
views,  evaluate  and  grade  the  options. 

Workshop  Framework 

The  workshop  mission  was  to  define  a  statement  that  the  Core  Team 
could  use  to  help  frame  a  Strategic  Plan.  In  this  respect,  the 
Core  Team  was  seen  as  the  "customer"  of  the  workshop. 

Initially,  a  report  on  each  of  the  key  business  drivers  was 
presented  and  discussed.  This  provided  a  framework  for  the 
subsequent  assessment  of  the  business  options.  After  this,  the 
full  set  of  business  options  generated  by  the  Core  Team  over  the 
preceding  months  was  presented  by  the  Core  Team.  At  this  point  the 
TEAMWORKER  system  was  used  as  part  of  a  structured  process, 
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involving  a  combination  of  discussion  and  polling,  to  determine  a 
sub-set  of  up  to  13  options  to  be  studied  more  intensively  at  the 
workshop,  and  then  to  become  the  subject  of  post-workshop 
activities.   This  was  called  the  "Screening  Process". 

The  next  stage  involved  a  number  of  small  teams  each  working  on  one 
of  the  sub-set  of  options.  Each  team  was  organized  to  have  at 
least  one  delegate  from  a  business  function.  Presentations  were 
then  made  by  each  team  in  a  plenary  session,  detailing  the 
advantages  and  disadvantages  of  the  business  option  studied  by  that 
team. 

Following  this,  the  TEAMWORKER  system  was  used  to  score  the  sub-set 
of  options  on  each  of  two  criteria;  the  relative  likelihood  that 
the  option  would  be  feasible  in  the  timescale  under  consideration, 
and  the  relative  commercial  impact  that  the  option  would  have  on 
the  organization  in  this  timescale.  This  was  known  as  the  "Mapping 
Process".  The  output  from  this  process  consisted  of  a  two 
dimensional  grid  showing  the  workshop's  evaluation  of  each  of  the 
sub-set  of  options  on  the  two  criteria. 

The  outputs  from  these  two  processes  were  used  to  formulate  the 
final  guiding  statement  to  the  Core  Team.  These  processes  are  now 
described  in  more  detail  below. 


The  Screening  Process 

Each  of  the  set  of  35  Business  Options  was  presented  by  selected 
participants.  The  details  of  these  options  cannot  be  listed  in 
this  paper,  but  included  items  such  as  "Chemical  Vehicles 
Development". 

Following  the  presentations,  and  some  further  clarifying 
discussions,  each  workshop  delegate  used  a  1  to  5  scale  to  score 
each  of  the  business  options.  The  business  option  considered  of 
most  importance  (in  the  opinion  of  a  particular  person)  was  given 
a  5,  and  the  option  considered  of  least  importance  given  a  1.  All 
other  options  were  then  given  a  unitary  value  of  between  1  and  5 
relative  to  the  individually  defined  most  and  least  important 
options.  This  approach  served  to  anchor  the  end  points  of  the 
subjective  scoring  scale  of  each  participant  independently. 

TEAMWORKER  was  used  to  collect  the  scores  and  feedback  summarized 
information  to  the  workshop  for  discussion  and  debate.  The  first 
feedback  screen  provided  to  the  delegates,  following  a  particular 
option  vote  is  shown  in  Fig.  2.  This  is  a  histogram  showing  the 
numbers  of  scores  obtained  for  each  value  on  the  1-5  scale,  the 
overall  average  and  standard  error,  and  the  level  of  confidence 
associated  with  statement  that  the  sub-sets  of  scores  from  those 
R&D  Function  members  were  significantly  different  from  those  of  the 
Business  Function  members.   The  average  used  was  not  a  simple 
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average  of  all  43  delegates,  but  the  mean  of  the  averages  of  the 
two  sub-groups.  This  form  of  average  was  subsequently  used  to  aid 
determination  of  the  options  to  be  considered  at  the  next  stage  of 
the  workshop,  and  the  Core  Team  did  not  want  the  larger  number  of 
R&D  delegates  to  have  a  more  than  equal  (quantitative)  influence  in 
the  process.  Histograms  of  the  type  shown  in  Figure  2  clearly 
showed  how  much  disagreement  there  was  in  the  workshop  for  each 
option.  In  addition,  the  calculated  confidence  level  indicated 
whether  there  was  substantial  disagreement  between  the  two  groups 
of  members.  A  feedback  screen  was  used  in  order  to  compare  the 
sets  of  R&D  and  Business  scores  (Fig.  3) .  Each  of  these  feedback 
screens  frequently  led  to  further  discussion  and  debate, 
particularly  when  significant  differences  were  revealed,  often 
resulting  in  rescoring  of  an  option  by  the  group. 

When  all  the  options  had  been  considered,  a  "High-Low"  chart 
showing  the  average  score  for  each  option,  together  with  the 
standard  error,  was  presented  to  the  group  (Fig.  4) .  Included  with 
this  chart  was  a  ^grey  area'  which  depicted  the  zone  where  the  cut- 
off for  the  sub-set  of  options  to  be  studied  further  was  located. 
This  zone  was  established  in  a  debate.  Further  discussion 
concentrated  on  this  grey  area,  with  some  rescoring  of  options  by 
the  group. 

The  outcome  of  this  process  was  a  defined  sub-set  of  12  options  for 
further  attention  at  the  workshop  (the  "Mapping  Process")  .  Several 
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other  options  were  considered  worthy  of  further  work  after  the 
workshop  (sometimes  because  there  was  not  yet  enough  information  to 
make  a  judgement) ,  and  other  options  were  recommended  to  be 
discarded. 

The  Mapping  Process 

The  42  delegates  were  divided  into  12  work  teams,  each  to  develop 
one  of  the  selected  options  established  in  the  Screening  Process, 
by  studying  the  feasibility  and  marketability  of  their  option. 
Each  team  had  a  minimum  of  3  and  a  maximum  of  4  members,  including 
one  Business  Function  Member,  and  spent  a  day  developing  their 
option,  and  preparing  literature  which  could  be  presented  to,  and 
discussed  and  evaluated  by  all,  the  other  workshop  members.  A 
series  of  presentations  was  given  by  each  team,  when  the  results  of 
team  deliberations  were  disseminated  to  the  rest  of  the  workshop. 

Following  the  presentations,  all  delegates  used  a  1  to  5  scale  to 
score  each  of  the  subset  of  options  on  each  of  two  criteria;  the 
relative  likelihood  that  an  option  would  be  feasible  in  the  given 
timescale  and  the  relative  impact  the  option  would  have  on  the 
organization  if  it  was  feasible.  As  with  the  Screening  Process, 
each  delegate  first  (privately)  gave  the  top  option  on  each 
criterion  a  score  of  5,  and  the  bottom  option  a  score  of  1.  All 
other  options  were  then  given  a  score  of  1  to  5,  relative  to  the 
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individual's  top  and  bottom  options  on  each  of  the  criteria  taken 
in  turn. 

TEAMWORKER  was  used  to  collect  the  scores  on  each  criterion.  The 
first  feedback  screen  showed  the  scatter  of  scores  on  the  two 
criteria  (Fig.  5) .  In  this  figure,  the  areas  of  the  dark  squares 
are  proportional  to  the  numbers  of  scores  in  each  zone  of  the  grid 
respectively.  This  gave  a  visual  indication  of  the  degree  of 
disagreement  existing  within  the  group.  It  was  also  easy  to  view 
the  histogram  of  scores  for  each  of  the  criteria  (Fig.  6)  .  The 
Average  is  the  simple  mean  of  the  two  sub-group  averages  (R&D  and 
Business) ,  and  the  Standard  Error  shown  is  the  standard  error  of 
this  mean.  These  feedback  screens  led  to  further  discussion  and 
debate,  particularly  when  significant  differences  were  observed, 
sometimes  resulting  in  rescoring. 

Once  all  the  options  had  been  scored,  a  two  dimensional  grid  was 
shown  in  order  to  display  the  position  of  each  option  on  the  two 
criteria,  using  the  averages  and  standard  errors  in  both  directions 
(Fig.  7)  .  The  more  important  options  were  those  with  a  high 
Relative  Likelihood  score  and  a  high  Relative  Impact  score,  placed 
towards  the  top  right  of  the  figure.  These  were  identified  as  the 
high-grade  options  in  the  advice  given  by  the  workshop  to  the  Core 
Team. 
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4.     Discussion  and  Conclusions 

The  TEAMWORKER  system  was  undoubtedly  useful  in  terms  of  providing 
a  degree  of  structure  to  a  complex  task  carried  out  by  a  large 
group  of  experts.  It  also  enabled  rapid  identification  of  areas  of 
strong  disagreement,  making  it  easy  to  prompt  relevant  debate.  A 
number  of  aspects  or  issues  which  arise  from  this  case  application 
are  identified  as  follows: 

(i)  The  Core  Team  wanted  to  use  a  scoring  approach  which 
was  easy  to  understand  and  implement,  so  a  1-5  scale 
was  adopted  in  order  to  score  each  option  rather  than, 
for  example,  a  pairwise  or  ratio  scale  approach. 

(ii)  Each  participant  was  asked  to  identify  independently 
their  personal  lowest  and  highest  rated  options,  and  to 
allocate  scores  of  1  and  5  to  these  respectively.  This 
ensured  that  the  full  scale  was  used,  and  serves  as  a 
basis  for  producing  a  ranked  list  of  options  from  each 
delegate. 

(iii)  The  scores  themselves  were  used  in  order  to  calculate 
average  and  standard  deviation  values.  The  rigour  of 
this  form  of  aggregation  is  questionable.  An 
alternative  approach  could  be  aggregate  rankings  rather 
than  scores.   In  discussion,  the  Core  Team  preferred  to 

14 


work  with  the  scores  in  terms  of  clarity  of  the 
feedback,  bearing  in  mind  that  this  was  intended  to 
generate  debate  rather  than  create  excessive  dependence 
on  a  mathematical  routine  and  associated  output. 

(iv)  The  group  process  achieved  a  high  level  of  sustained 
attention  and  application,  involving  all  the  delegates 
over  several  days. 

(v)  The  feedback  screens  frequently  resulted  in  verbal 
expressions  of  surprise,  typically  concerned  with 
differences  of  judgement,  and  easily  served  to  generate 
focussed  debate. 

(vi)  Re-scoring  after  feedback  and  discussion  did  not 
invariably  reduce  differences  of  opinion,  but 
nonetheless  opinions  frequently  changed  at  these 
stages. 

(vii)  There  was  an  occasional  review  of  the  definitions  of 
options,  especially  during  debate  following  feedback. 

(viii)  The  workshop  resulted  in  the  group  reaching  a 
collective  decision:  a  defined  short-list  of 
options,  to  each  of  which  a  small  group  was  assigned 
for  further  indepth  review  and  assessment  after  the 
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Workshop.  There  appeared  to  be  commitment  to  this 
on-going  work,  perhaps  as  a  result  of  the  high  level 
of  involvement  of  the  participants. 

(ix)  The  workshop  was  a  collective  highlight  or  episode  in 
an  on-going  process  of  technology  assessment  and 
strategic  planning,  rather  than  a  start  or  finish  in 
its  own  right. 

We  conclude  this  paper  with  a  quotation  made  by  the  organizing 
officer  shortly  after  the  event:  "We  might  have  had  some  other, 
more  conventional,  meeting  but  I  don't  think  it  would  have  been 
nearly  as  productive". 
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Figure  1.  TEAMWORKER  Feedback  System 
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Figure  2.  Histogram  of  Scores  for  Complete  Group 
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Figure  3.  Histogram  of  Scores  for  R&D  and  Business  Groups 
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Figure  6.  Histogram  of  Scores  for  Relative  Likelihood  and  Relative  Impact 
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Figure  7.  Grid  Showing  Average  Scores  and  Standard  Errors  for  Each  Option 
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